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ABSTRACT 

Investigators completed a series of studies of data 
reported in Tennessee's 1988-89 school district cards, and compared 
these with 1990-91 school district report cards. The relationships 
among 15 school district variables and relationships between each 
variable and student outcomes were determined. The 1990-91 report 
cards contain results of the new Tennessee Comprehensive Assessment 
Program. Several conclusions from the previous study were reinforced 
by the current one. Policymakers at all levels need to consider that 
few of the individual inputs commonly associated with student 
achievement have much impact on student performance. With the 
exception of student attendance (and perhaps per pupil expenditure), 
treatment of any isolated variable will have little effect. It is 
urged that consideration be given to collecting, reporting, and 
analyzing data on such things as school organization, school culture, 
student motivation, parental involvement, instructional 
methodologies, curriculum features, and other factors. Bui lding~l evel 
results appear more useful than do district-level results, and report 
cards appear to be only as good as the assessments used to determine 
student performance. Ten tables and 1 figure summarize findings, and 
seven appendixes contain seven additional tables. (SLD) 
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BACKGROUND AND PROCEDURES 

The investigators have completed a series 
of studies of the data reported in Tennessee's 1988- 
89 and 1990-91 school district report cards. This 
report focuses on results on the analysis of 1990-91 
data with comparisons, whore appropriate, to 
findings of the earlier studies. Of particular 
importance are the analyses of the relationships 
between 15 school district characteristics 
(independent variables) and mean student 
outcomes (average achievement scores used as 
dependent variables) at the system level, school 
level (i.e., elementary, middle, high), and individual 
grade levels (2nd - 8th and 10th). Measures of 
student outcomes were results from Tennessee's 
Comprehensive Assessment Program (TCAP) and 
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the Tennessee Proficiency Test (TPT). 

Two correlation procedures (Guttman's 
partial correlation and coefficients of determination) 
were used to generate the data used as the basis for 
the primary analyses. Additional study procedures 
included the rank ordering of school districts by Mean 
Student Outcome (MSOs) and the computation of z- 
scores to assist in trend analysis of MSO. 
SELECTED FINDINGS 

Among the more interesting findings of the 
study were the following: 

1. At the school system level, student attendance 
(13.3%) and the expenditure per pupil (9.4%) 
have the largest impact on student outcome (see 
Table 1, p. 3). However, these influences did not 
have the same impact at ail school and grade 
levels. 

2. School system level characteristics having major 
influence on student performance at all levels 



ERLC 



BEST COPY AVAILABLE 



were percent of students receiving free/reduced 
cost lunches (9.4%) and percent of special 
education and/or Chapter I students in the district 
(6.5%), school or grade level (see Table 2&3). 
Again, these influences were not consistent at all 
school levels and grade levels. 

3. Tennessee report card factors did not influence 
student achievement in the same way at all 
school-levels and grade-levels. Some school- 
level factors (percent student attendance, 
percent oversized classes, percent free/reduced 
lunches, and expenditure per pupil) and grade- 
level factors (percent of student attendance, 
expenditure per pupil, percentage of Career 
Ladder teachers, percentage of free/reduced 
lunches) demonstrated dramatic upward or 
downward shifts in influence {see Tables 2 & 3}. 

4. Some highly regarded factors such as 
percentage of oversized classes, average 
professional educator salaries, per capita income 
in the school district, percent of enrollment 
change, percent of special education, and size of 
the school district had relative^ little Impact on 
student achievement. 

5. Together, the 15 district factors studied 
accounted for less than 50 percent of the total 
influence on achievement at any schooMevel 
(/.a, elementary, middle, high school} or grade 
level (Table 2). When the school levels were 
examined, the greatest impact of the combined 
factors came at the high school level (41%) and 
the smallest impact occurred at the middle 
school level (35%). At individual grade-levels. 
the greatest impact of the combined 15 factors 
was at the 4th grade level (48%), and the least 
impact was at the 6th grade level (1 9%). 

POLICY RELATED CONCLUSIONS AND 
IMPUCATIONS 

Several conclusions and implications 

extrapolated from the findings are worthy of 



discussion by report card developers and 

policymakers at the local, state and national levels: 

1. Improvement in student academic performance 
will require that all controllable factors in a school 
or school district receive attention, not just one or 
two (e.g., class size, per pupil expenditure, 
professional salaries, eta) that are high on the 
agenda of one or another stakeholder groups. 

2 Improvement in student academic performance 
requires that we identify the factors that aocount 
for the remainder of the influence on student 
outcomes. This means that school districts must 
collect data on school climate and culture, 
instructional methods, school organization, 
parental involvement, student attitudes, and other 
factors. Only analyses of the relationships 
between these factors and student outcomes can 
provide us the additional information the we need. 

3. Report cards containing only system level data 
are of little value in determining what can or 
should be done to bring about improvement in 
student performance. The interactions between 
district and school characteristics and student 
outcomes are complex, and they vary greatly by 
school level and grade level. 

4. The findings of this study suggest that 
unquestioning equal treatment of schoois within 
a district and students at all grade levels in terms 
of expenditures, placement of teachers, class 
size, and other matters may actually create 
educational inequities. Provision of equal 
educational opportunity may very welt require 
dissimilar allocations of human and financial 
resources. 

5. Findings from this study and its predecessors 
suggest that the tests/assessments chosen to 
measure student performance are critical factors in 
determining what is currently working in a particular 
context and what needs "fixing." The data 
generated from such analyses are only as good as 
the assessments administered to students. 



Infusion of money is clearly not the single 
prerequisite to improved student academic 
performance. In this and the previous studies, 
too many school districts demonstrated student 
outcomes inconsistent with expectations based 
on financial conditions in the district. While 
financial recources are important to student 
achievement, there is evidence to suggest that 
other factors are equally, and in some cases, 
more important. 

The purpose(s) of a school report card should 
be established before the context and format of 
the report card are determined. Simple 
reporting of the status of a series of factors 
within a school or district can be done in several 
ways, using a variety of information. However, 



the development of a report card that will assist 
educators, policymakers and stakeholder 
groups in targeting areas and strategies for 
improvement requires quite different content and 
format. 

The corresponding research papers are available 
by sending $10 (cover copying and postage) and 
contacting: 

Educational Research and Consulting: 

Dr. G.C. Bobbett, 8325 Richland Colony Rd., 
Knoxville, TN 37923 
(615) 691-4253 



Table 1 . Comparison of Influence Exerted on Student Academic Outcomes by 
School District Characteristic in 1988-89 and 1990-91. 

Percentage of Influence (district level) 



District Characteristics "988-89 1990-91 

• County Per Capita Income 0.4 0.4 

• Average Professional Salaries r57S1 0.1 

• Expenditure Per Pupil 0.0 I 9.4 1 

• Average Daily Membership 2.8 0.9 

• % Student Attendance PTO 

• % Oversized Classes 0.6 3.1 

• % Free/Reduced Lunches I 6 0! I 4.7i 
•% Career Ladder II & III 0.2 3.1 

• Number of Schools in District 0.4 

• % Enrollment Change 1 .5 

• % Regular Diplomas 1 .5 

• % Honors Diplomas 0.2 

• % Students enrolled in 

Vocational Education 2.9 

• % Students in 

Special Education 0.2 

• Percentage of Chapter 1 Students I 6.5 1 

Total Percentage of Influence 26.5 48.2 



Box> 4% Percentage of Influence 
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Table 2. Comparison of Influence Exerted on Student Academic Outcomes by 
School-level, 1990-91 Tennessee Report Card data. 

Elementary Middle Secondary 



District Characteristics 



County Por Capita Income 

• Average Professional Salaries 

• Expenditure Per Pupil 

• Average Daily Membership 

• % Student Attendance 

• % Oversized Classes 

• % Free/Reduced Lunches 

• % Career Ladder II & III 

• Number of Schools in District 

• % Enrollment Change 

• % Regular Diplomas 

• % Honors Diplomas 

• % Students enrolled in 

Vocational Education 

• % Students in 

Special Education 
Percentage of Chapter 1 Students 



Total Percentage of Influence 



0.4 


0.0 


1.0 


0.3 


111-21 


I 9,1 1 


0.1 


0.2 


I 6.7I 


I 5,9l 


3.4 


2.8 




2.3 


3.2 


I 4,9| 


0.2 


0.0 


0.3 


0.3 


1.1 


1.9 


0.3 


1.5 


0.8 


1.0 


1.5 


0.1 


2.1 


I 6.0 I 


39.6 


35.3 



System 



0.6 


0.4 


2.7 


0.1 


0.4 


I 9.4I 


5.3 


0.9 


13.6 


113.3! 


0.2 


3.1 


0.3 


I 4.7| 


0.0 


3.1 


4.51 0.4 


3.5 


1.5 


0.2 


1.5 


0.4 


0.2 


K§ 


I 2.9 


0.0 


0.2 


1 4.7 


I I 6.5 1 


40.9 


48.2 







Table 3. Comparison of Influence Exerted on Student Academic Outcomes by 
Gradn-Ievel, 1990-91 Tennessee Report Card data. 



District Characteristics 



C 
CM 



"2 

CO 



5? 



to 



as 



Si 



E5. 



County Per Capita Income 2.3 0.1 0.0 

Average Professional Salaries 0.0 0.9 L 1.7 

Expenditure Per Pupil 1.6 QT23 CSZI 

Average Daily Membership 0.5 0.2 0.9 

Student Attendance 11321 3.5 

% Oversized Classes 2.0 1 .6 

% Free/Reduced Lunches 0.8 2.5 

% Career Ladder II & III [ODD 0.4 0.0 

Number of Schools in District 0.4 0.7 0.7 

% Enrollment Change 0.0 0.4 1.7, 

% Regular Diplomas 0.1 0.2 

% Honors Diplomas 0.8 0.5 

% Students enrolled in 

Vocational Education 0.0 0.0 1 4,1 1 0.6 
% Students in 

Special Education 2.1 0.2 1.9 
Percentage of Chapter 1 

Students 0.1 3.9 1.0 



^2 



0.0 
0.8 

QM1 

0.2 
0.0 
0.5 

ESI 

1.5 
0.3 
1.5 
1.1 
0.0 



1.6 
1.1 



0.6 
0.1 

0.0 
1.1 
2.2 
2.8 
2.0 
0.0 
0.4 
0.3 
1.8 

1.0 

0.5 

2.4 



0.1 
0.7 
[Ml 
0.0 

roi 

2.7 
1.4 

BE 

0.3 
0.3 
0.8 
1.3 

2.2 

0.8 



Total Percentage of Influence 39.9 26.9 48.3 31 .2 19.4 



2.4 
30.6 



0.7 
0.1 

1421 



IAZ 
0.6 
0.3 
2.8 
0.3 
1.9 
3.4 
0.1 

0.0 

0.7 

Da3 



0.0 
2.9 
0.6 
0.9 
3.0 
1.3 
0.3 
1.5 
0.6 
0.6 
1.0 
0.0 

2.2 

2.5 

2.0 



26.7 19.5 



Box> 4% Percentage of Influence 
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AN ANALYSIS OF REPORT CARDS ON SCHOOLS: HOW COMMUNITY/SCHOOL 
CHARACTERISTICS IMPACT STUDENT OUTCOMES 1 



I. INTRODUCTION 

In 1991 , the investigators completed a series of studies of the data reported in 
Tennessee's 1988-89 school district report cards. In those studies which have been 
reported in several papers (Bobbett, et al., 1992a, 1992b), the relationships among 
eight school district variables {average attendance, average professional salaries, 
county per capita income, expenditure per student, average daily membership, percent 
of oversized classes, percent of students on free or reduced lunches and percentage of 
professional educators on upper Career Ladder levels II and III) were examined, and 
the relationships between each variable and student outcomes were determined. 

The study reported herein is an extension of the previous study. In this 
investigation, 1990-91 report card data were used. Because of that, it was possible to 
revisit some of the relationships in the previous study and to gain new insights because 
of modifications in Tennessee's report cards from 1989 to 1991 . 

In 1990-91, Tennessee brought "on line" its new Tennessee Comprehensive 
Assessment Program (TCAP), thereby creating a new set of student outcome 
measures. Further, TCAP results were reported in greater detail than were previous 
outcome data. Report cards now report TCAP assessment results at more grade levels 
within the school districts than was previously done. 

The 1990-91 report cards also added more school district characteristics; thereby 
enabling the investigators to expand their analyses from 8 to 15 variables. The seven 
added variables include number of schools in the district, percent of enrollment change, 
percent regular diplomas awarded, percent honors diplomas awarded, percent 
vocational students, percent special education students, and percent Chapter I students. 

While certain comparisons in the results of the two studies can be made, some 
findings cannot be compared because of the differences in the outcome measures used in 



1 . This paper includes material presented at the annual meetings of SRCEA (11/92) 
and MSERA (11/92), and extends the analyses of data to produce several interesting 
new findings. 

1 
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the different years and because no comparable data were available in certain areas in the 
1 988-89 report cards. Tables 1 and 2 present a school district report card as it appeared in 
1988-89, and Tables 3 and 4 represent a 1990-91 school district report card. 

II. METHODOLOGY 

The 1 990-91 report cards provided test results for grades 2 through 1 0. The 
investigators conducted analyses at the school system/district, school-level and 
individual grade-levels. School-level analyses organized data at four levels: 
elementary (grades 2-5), middle school (grades 6-8), high school (9-10), and system- 
level (grades 2-10). 

Mean student outcomes (MSOs) were created (by converting reported scores to 
Z scores and computing their means) for each level by combining TCAP data for the 
grades defined within the particular level. For the high school level, the MSO was 
created by combining 1 0th grade TCAP data with the scores reported for the 9th grade 
Tennessee Proficiency (TPT). These MSOs were treated as dependent variables, as in 
the analysis of 1988-89 report card data. The 15 school district characteristics studied 
were treated as independent variables that influence student outcomes. To guide the 
study ten research questions were developed: 

1 . How do school district characteristics currently reported relate to student 
academic achievement results? 

2. Are there differences in the relationships between dependent and independent 
variables at different school levels (elementary, middle, high jchool, system)? 

3. Are there differences in the relationships between dependent and independent 
variables at different grade levels (2nd, 3rd, 4th, etc.) 

4. How do reported school characteristics relate to each other? 

5. When rank ordered on the basis of student outcomes, how do school districts 
within the state perform in terms of reported school and community 
characteristics? 

6. Do the reported school district characteristics appear to represent ail or most 
factors that influence student academic achieverhent? 

7. Is there evidence of major change in student academic performance from one 
school level to anotner within school districts? 

8. When academic achievement is treated as scores on two separate test batteries 
(TCAP and TPT), are patterns of influence changed? 

9 What differences in relationships among variables exist when test results of 
special education students are includea in the analyses? 

2 
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Table 1 . Testing Information For Widget City Schools (1 988-89 Report Card Data) 



Testing Information 
for Widget City 


Grade 
Level 


1987-88 


1988-89 


State 
Average 


Basic Skills First 




3 


90 


88 


80 


Reading 


6 


82 


80 


77 




8 


9? 


91 


81 


Achievement Test 

(percent score) 




3 


91 


90 


82 


Math 


6 


67 


71 


66 




8 


77 


84 


66 


Stanford 
Achievement 
Test 

(Stanine score) 

7-9 = High 
4-6 m Average 
1 -3 - Low 


Reading 


2 


6 


7 


6 


5 


6 


6 


5 


7 


6 


6 


5 


Math 


2 


7 


8 


6 


5 


7 


6 


6 


7 


7 


7 


5 


Spelling 


2 


6 


7 


6 


Language 


5 


6 


6 


5 


7 


6 


6 


5 


Environment 


2 


7 


7 


6 


Science 


5 


6 


7 


6 


7 
f 


6 


6 


c 
O 


Listening 


2 


7 


7 


5 


O 


6 


6 


5 


f 


6 


6 


5 


Social 
Science 


5 


6 6 


5 


7 


6 


6 


5 


Stanford Test of 
Academic Skills 

(TASK 2) 

7-9 = High 
4-6 = Average 
1-3«Low 


Reading 


9 


6 


6 


5 


12 


6 


6 


5 


Math 




6 


6 


D 




6 


6 


c 
D 


English 


9 


7 


6 


5 


12 


6 


7 


5 


Science 


9 


7 


6 


5 


12 


6 


6 


5 


Social 
Science 


9 


5 


6 


5 


12 


6 


5 


5 


Tennessee Proficiency Test 

(% Students Passing) 


Language 


9 


88 


32 


78 


Math 


9 


95 


98 


90 


Both 


9 


86 


91 


76 



Table 2. System Information for Widget City Schools (1988-89 Report Card Data). 



System Information 
for Widget City 


Grade 
Level 


1986-87 


1987-88 


1988-89 


State 
Average 


Number of Schools 


X 1 9 

rV I d. 


5 


5 


5 


12 


Average Daily Membership 


K-12 


3,291 


3,394 


3,372 


5,874 


% Student Attendance 


K-12 


95.7 


95.3 


95.1 


93.6 


% Enrollment Change 


9-12 


-13.0 


-16.1 


-15.2 


-24.7 


% Oversized Class 


K-12 


1.2 


1.4 


2.3 


3.8 


% of Students on Free or Reduced Price Lunch 


K-12 


23 


21 _ 


21 


42 


Expenditures per pupil 


K-12 


$2,718 


$3,299 


$3,501 


$3,304 


County Per Capita Income 


K-12 


it 


ii 


$12 f 819 


$12,878 


% Elementary Schools Accredited by SACS 


K-8 




100.0 


100.0 


29.1 


% Secondary Schools Accredited by SACS 


7-12 


100.0 


100.0 


100.0 


64.9 


Professional Educator Information 












% Professionals on Career Ladder Levels II & III 


K-12 




21 .9 


25.6 


14 ft 


Average Professional Salary 




1 Qfl fin 


$26,085.44 






Student Information 












% Diplomas 
Granted 


Regular 


12 


90.6 


68.7 


75.8 


81.8 


Honors 


12 


49,6 


26.7 


20.0 


8.5 


Special Education 


12 


1.8 


1.4 


1.5 


1.9 


Certificate of Attendance 


12 








0.9 


Seniors not Receiving 
Diploma in Spring 
Graduation 


12 


2.7 


3.2 


2.7 


6.9 


% Students in Vocational Education Courses 


7-12 


33.0 


40.9 


41.0 


45.5 


% Students in Special Education 


K-12 


12.1 


11.3 


12.1 


14.2 


% Chapter 1 Students 


K-12 


13.3 


15.5 


12.4 


11.9 



4 



Table 3. Testing Information For Widget City Schools Too (1990-91 Report 
Card Data. 

Widget Too Schools 









Year 


2 


3 


4 


5 


6 


7 


8 


10 


Reading 


State Avq. 


na 


na 


na 


na 


na 


na 


na 


na 




1990-91 


7 


6 


6 


6 


6 


7 


7 


6 


TENNESSEE 


GRADE 


COMPREHENSIVE Language 

ASSESSMENT 

PROGRAM (TCAP) 

Math 


Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avg. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


6 


6 


6 


6 


7 


6 






Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avq. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


7 


7 


6 


6 


7 


7 


7 




GR 


ADE 


wvlwl Ivv 


Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avq. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


7 


6 


6 


7 


6 


6 


Social 
Studies 


GRADE 


Year 


2 


3 


4 


5 


6 


7 


8 


10 


State Avg. 


na 


na 


na 


na 


na 


na 


na 


na 


1990-91 


7 


6 


7 


6 


6 


6 


6 


6 








Grade 9 




TENNESSEE La "9" a 9e 


Year 


With Special Ed. 


Without Special Ed. 




State Avg. 


na 


na 


1990-91 


90 


91 


PROFICIENCY 

TEST (TPT) Mathematics 






Year 


With Special Ed. 


Without Special Ed. 




State Avq. 




na 


1990-91 


98 


98 








Both 


Year 


With Special Ed. 


Without Special Ed. 




State Avq. 




na 


1990-91 


88 


90 









Testing Information 

Students in Tennessee are given two types of tests. 

Students were introduced this spring to the 
Tennessee Comprehensive Assessment Program 
(TCAP). This program mandates a customized, norm 
referenced and criterion referencad test for grades 2 
through 8, a norm referenced test for grade 10, and the 
Tennessee Proficiency Test. 

The customized test will allow each teacher to 
assess progress of students during the school year with 
a minimum amount of testing time. The program will 
generate consistent types of test scores from grade to 
grade. The norm referenced data will allow longitudinal 
status of individual, school, system, and state growth in 



order to evaluate and improve programs and curricula. 
The criterion referenced data will report the mastery , 
partial mastery, and non-mastery of tested domains for 
each school year. Although the objectives for the 
Tennessee Proficiency Test has been updated, the rules 
and regulations governing the test will remain the same. 

The Tennessee Proficiency Test measures 
minimum skills in mathematics and language arts. 
Students must achieve a passing score of 70 percent 
correct on both the math and language arts tests in order 
to fulfill one of the requirements for receiving a regular 
diploma. Students take the test for the first time in the 
ninth grade. 
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Table 4. System Information for Widget City Schools (1 990-91 report card data). 



Widget Too 



System Information 


Grade 
Level 


1988-89 


1989-90 


1990-91 


State 
Averaae 


Number of Schools 


K-12 


5 


5 


5 


na 


Average Daily Membership 


K-12 


3,372 


3,9290 


3,436 


na 


% Student Attendance 


K-12 


95.1 


95.8 


95.6 


na 


% Enrollment Change 


9-12 


-15.2 


-12.1 


-20.1 


na 


%Oversized Classes 


K-12 


2.3 


1.4 


1.5 


na 


% of Students on Free or Reduced Lunches 


K-12 


21.0 


22.0 


23.0 


na 


Expenditure per Pupil 


K-12 


$3,501 


$3,942 


$4,073 


na 


County Per Capita Income 


K-12 


$12,819 


$13,662 


$14,192 


na 


% Elementary Schools Accredited by SACS 


K-8 


100 


100 


100 


na 


% Secondary Schools Accredited by SACS 


7-12 


100 


100 


100 


na 


Professional Educator Information 




% Professionals on Career Ladder II and HI 


K-12 


25.6 


28.6 


30.8 


na 


Average Professional Salary 


K-12 


$30,804.37 


$31,590.60 


$33,753.00 


na 


Student Information 




% Diplomas 
Granted 


Regular 


12 


75.8 


73.4 


79.5 


na 


Honors 


12 


20.0 


22.0 


18.6 


na 


Special Education 


12 


1.5 


0.9 


1.0 


na 


Certificate of Attendance 


12 




.09 




na 


Seniors not Receiving 
Diploma in Spring 
Graduation 


12 


2.7 


2.8 


1.0 


na 


% Students in Vo 


rational Education Courses 


7-12 


41.0 


41.3 


39.3 


na 


% Students in Special Education 


K-12 


12.1 


12.6 


13.6 


na 


% Chapter 1 Students 


K-12 


12.1 


12.6 


8.7 


na 



Othar information: 

Parcant of Student In Attandanc* (%SA). 

This figure shows the average percent of student in 
attendance daily in your school system for the 1 990 
91 year. 

Parcant Enrollment Chang* (%EC). This 
figure shows the percent change in a group of 
student who started in the ninth grade four years 
ago and should have completed the welfih grade 
this year. It is a four year average. Decreases 
happen when students drop out of a school, move 
away, graduate early, fail a year, or leave school for 
other reasons not listed. 

Parcarrt of Ovareizad Class** (%OC). This 
figure shows the percent of classes in all grade 
levels which had waivers for being over the 
maximum class size. Maximum class sizes in 
Tennessee are 25 for grades K-3; 28 for grade 4, 90 
for grades 5-6; 35 for grades 7-12; 23 for vocation. 

Parcarrt Students on Fra* or Raducad 
Lunch** (%FRL): Students whose family income 
meets certain criteria are eligible for free or reduced 
price lunches. This figure shows the percent of 
student sin your school system who receive free or 
reduced price lunches. 

Expandftur* par Pupil (EPP). This figure 
shows the average number of dollars spent for each 
pupil in average oaily attendance for your school 
system. 

County P*r Capita lncom*(CCI): This figure 
represents the per capita personal income for Die 
county in which your school system is located. The 
most recent figures available from the U.S. Bureau 
of Economic Analysis are for 1 988. 



Paroant EJ*m*ntary/S*condary School* 
AccradHad by SACS (%ES): Schools may elect to 
seek accreditation from the Souther Association of 
College and Schools (SACS) in addition to receiving 
state approval. This agency recognizes quality 
schools, maintains a list of accredited schools and 
requires a continuing school improvement program. 

Parcarrt Profa**ional* on C*r**r Laddar 
Lavata II and III (%CL): This figure show the 
percent of professional staff In your school system 
who have met the standards for Career Levels II 
and III. These are the upper rungs of Tennessee's 
Career Ladder program. The number includes 
regular classroom teachers, guidance counselors, 
librarians, and administrators. 

Avantgo Profa**k>n*J Salary (APS): This 
figure shows the estimated average salary for ail 
certificated personnel in your school system. 

Dirjtoma* Grantad: These figures show the 
percent of the twelfth grade class receiving different 
types of diplomas. Some school systems nave 
requirement that may exceed these standards/ 
Tennessee students may receive four kinds of 
diplomas: 

High School Diploma (D-KS): Awarded to 
students who (a) earn 20 units of credit, (b) make 
passing scores on aH components of the 
Proficiency Test and (c) are satisfactory records 
of attendance and conduct 

Honor* Diptana (D-HC): School systems 
may offer an optional diploma to students who 
meet increased requirements established by the 
State Board of Education, The requirements 
include accelerated English, math, science and 



social students, and a 3.0 grade point average. 

Special Education Diploma (D-SE): 
Awarded to students who have safotactorily 
completed an Individualized Education Program 
and who have satisfactory records of attendance 
and conduct, but who have notpassed all 
components of the Proficiency test 

Cartiftcata of Attandanc* jt>CA): Awarded 
to students who have earned 20 units of credit 
and who have satisfactory records of attendance 
and conduct, but who fail to meet Proficiency 
Test standards. 

Studants Not Rac*lvlng Diploma in Spring 
Graduation (D-NR): This figure represents 
students who will receive their diplomas after 
completing summer school or who failed to 
complete nigh school. 

Parcarrt or Student* In Vocational Education 
Coursa* (%VO): This figure shows the percent of 
the school system's average daily membership 
enrolled in one or more vocational education 
courses. Students enrolled in more than one 
vocational courses are counted only once. 

P*rc*nt of Studanta In Spccfai Education 
(%SE): This figure show the rjercent of students in 
your school system who are receiving special 
education services. 

P*re*nt of Chaptar 1 Studant* (%CH1): 
Chapter 1 is a federally funded program to assist 
students In the areas of reading and mathematics. 
This figure shows the percent of student receiving 
services under Chapter 1 . 



10. How do the results of this study compare with the results of the investigation 
using 1 988-89 report card data? 

Five of the ten questions replicate questions posed in the previous study; items 2, 3, 7, 

9, and 10 are new questions representing the capacity available in the 1990-91 report 

cards to analyze data at several levels within the school districts and the capacity of the 

current study for comparison with the earlier study results. Question 8 is a modification 

of a question posed in the earlier study, because only two test batteries (rather than 

three) were used in the current analysis. 

As in the earlier study, 120 of 138 districts reported comprehensive scores on 
both TCAP and TPT. These districts (120) constitute the sample for analysis. 

Twenty school district characteristics were actually reported in the 1990-91 
report cards. In responding to research Question #1 , the investigators first evaluated all 
characteristics to determine their value as independent variables. A Kaiser test of 
variable sample adequacy was applied to each variable at each level (elementary, 
middle, high school, and system). Five characteristics were eliminated from further 
study: percent elementary schools accredited by SACS, percent high schools 
accredited SACS, diplomas granted in special education, certificates of attendance 
granted as diplomas, and seniors not receiving diplomas in Spring graduation. 
Appendix A presents the results of this analysis. 

Two correlation procedures were used to generate a response to research 
question #1 . A Pearson Product Moment correlation enabled comparison of variables, 
and Guttman's partial correlation allowed the researchers to develop percentages of 
influence to assess relationships between independent and dependent variables. 

To answer research question #2, the correlations (Pearson and Guttman's) were 
generated for each independent-dependent variable relationship at each of the four 
defined school levels. 

Question #3 again required the use of Guttman's partial correlation procedure. In 
this analysis, correlations between each of the 15 school/community characteristics and 
Mean Student Outcome (MSO) at each grade level, two through ten were computed. 

Research question #4 was answered by computing correlations among 
independent variables. A coefficient of determination (r 2 ) showed the levels of 



interaction between categories (variables). 

Research question #5 required the rank ordering of school districts within the 
sample by system MSO. Comparisons of rankings at all school levels (elementary, 
middle, secondary) could be made. Only the top 1 0 and bottom 1 0 districts in the 
rankings are reported. 

Research question #6 required no further statistical analyses. The partial 
correlation coefficients and related percentages of influence previously developed 
provided the necessary data. 

To answer research question #7, changes in MSO upward and downward of one 
standard deviation from school level to school level were first computed using Z-scores 
as the basis for the computation. To further clarify the results, school-level rankings 
were developed. 

For research question #8, the investigators applied the Guttman partial 
correlation procedure to the relationship between each independent variable and each 
of the two test scores (TCAP and TPT) used in generating the high school MSO. 

Research question #9 required application of the statistical procedures 
previously used to the relationships between each of the 1 5 school district 
characteristics and TPT test scores for grade nine under two conditions: with and 
without special education student's scores. 

Research question #10 allowed the investigators to compare and contrast 
findings from the 1988-89 study and the 1990-91 study, wherever comparisons 
appeared to be valid. Some results could not be compared because different test 
batteries were used in the two different years. 

Question #11 was used as a means of focusing conclusions and implications. 
Report cards on schools and the data included in them generate policy discussions. 
The findings of this study when added to those of the earlier one should be useful to 
policymakers at all levels. 

III. FINDINGS 

Findings are reported in two ways: (A) a descriptive analysis of the 1 20 school 
districts used in the study, and (B) responses to the research questions. 
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A. Descriptive Analysis of School Districts 

A profile of the 1 20 school districts qualifying for inclusion in the study was 
developed. For each category, the report card (state) mean score, standard deviation 
(SD), number of schools submitting data and ranges of scores or numbers were 
compiled. Table 5 presents the profile. 

1. System Information 

All school districts in the sample (1 20) reported scores for TCAP and for the TPT. 
When special education students were included in the TPT results, 87.1 percent of all 
students passed the language test, 90.8 percent passed math, an'j 84.0 passed botn. 
When special education students were excluded from the report, 92.2 percent of the 
students across the state passed the language test; 94.9 percent passed the 
mathematics test and 89.7 percent passed both tests. 

Most of the 1 20 school districts studied reported all data for the 20 report card 
categories. The exceptions: 103 reported percentage of oversized classrooms; 48 
reported percentage of elementary schools accredited by SACS; 83 reported 
percentage of secondary schools accredited by SACS; 119 reported percentage of 
professionals on Career Ladder II & III, and 66 reported percent of certificate of 
diplomas awarded. The statewide profile shows approximately 13 schools per district 
with an average daily membership of 6,624 students. In 1 990-91 , student attendance 
averaged 94.4 percent statewide; enrollments in the districts decreased from the 
preceding year by an average of slightly more than 23 percent. In these districts, . 
approximately 4.4 percent of all classes exceeded state prescriptions for class size. 
Almost 42 percent of all students state wide received free or reduced lunches. Per 
pupil expenditures averaged $3,442 per district, and county per capita income 
averaged $12,371. 

2. Professional Educator Information 

Approximately 17 percent of all Tennessee educators had achieved Career 
Ladder Levels II or III by 1 990-91 , and average professional salary was $27,465. As 

9 



Table 5. A Report Card Profile of 120 Tennessee School Districts Sampled, 
1990-91 data. 



Tennessee Proficiency Test (TPn 
With Special Education 

Language 
Math 
Both 

Without Special Education 

Language 
Math 
Both 

System information 

Number of Schools 
Average Daily Membership 
% Student Attendance 
%Enrollment Change 
% Oversized Classes 
% ReeorRedJoedLLixhes 

Expenditure Pupil 
County Per Capita Income 
% El. Schools accredited by SACS 
% Sec. Schools accredited by SACS 

Professional Educator information 

% Career Ladder II & III 
Average Professional Salary 

Student Information (% Dip lomas Granted^ 

Regular 
Honors 

Special Education 
Certificate of Attendance 
Seniors not receiving 

Diploma in Spring Grad. 
% Students in Vocational Ed. Classe 
% Students in Special Ed. 
% Chapter 1 Students 











District 


cn 


Max 


Mln 
Mill. 




mean 


6.9 


99 


66 


120 


87.1 


5.8 


100 


68 


120 


90.8 


8.3 


99 


58 


120 


84.0 


5.6 


100 


72 


120 


92.2 


4.6 


100 


74 


120 


94.9 


5.6 


100 


72 


120 


89.7 


20.1 


161 


1 


120 


12.9 


12,415 


103,987 


378 


120 


6,624 


1 


97.4 


91.2 


120 


94.4 


9.4 


3.6 


-48.3 


120 


-23.0 


3.5 


23 


0.2 


103 


4.4 


142 


85 


10 


120 


41.7 


$532 


$5,312 


$2591 


120 


$3,442 


$2,257 


$22,097 $8,081 


120 


$12,371 


34.8 


100 


3 


48 


60.4 


23.1 


100 


25 


83 


85.3 


6.0 


42.5 


6.8 


A A C\ 

119 


ACQ 

16.8 


$2,960 


$36,505$23,262 


120 


$27,465 


9.2 


98.7 


56.3 


120 


80.4 


7.0 


41.7 


1 


102 


13.7 


1.6 


8.6 


0.4 


107 


2.4 


0.7 


2.9 


0.1 


66 


.9 


4.3 


21.3 


0.3 


97 


6.5 


S 13.7 


98.8 


19.8 


120 


47.6 


3.9 


28.8 


8.2 


120 


16.4 


8.1 


47.5 


2.6 


120 


16.2 
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few as 6.8 percent of the teachers in a district and as many as 42.5 percent had 
achieved upper Career Ladder status. Average salaries reported ranged from $23,262 . 
to $36,505. 

3 Student Information 

Eighty percent of all diplomas awarded in the state in 1990-91 were Regular 
diplomas; almost 14 percent were Honors diplomas; slightly more than 2 percent were 
Special Education diplomas, and about 1 percent of all students leaving school were 
granted certificates of attendance. More than 6 percent of students graduating did not 
receive their diplomas during spring graduation. 

Almost 48 percent (47.6%) of Tennessee's students were enrolled in vocational education 
classes during the year investigated. Slightly more than 1 6 percent were special education 
students, and another 1 6 percent were participants in Chapter 1 programs. 

4. Comparison of selected 1990-91 data with 1988-89 data. 

A few comparisons of data from the 1990-91 profile (see Table 5) with data from 
1988-89 (see Table 6) are useful. Passing rates for the TPT had risen substantially in 
language (M=76%, 92%, respectively), and in passage of both language and 
mathematics tests by 1991 (M=76%, 84%, respectively). 

Between 1989 and 1991, average per pupil expenditures rose about $100, and 
average county per capita income had fallen by about $500. Average professional 
salaries of educators had increased about $700. The percentage of students receiving 
free or reduced lunches remained static at approximately 42 percent, and the percent of 
oversized classes dropped only 3 tenths of one percent. 

B. Findings Pertinent to Research Questions 

1 . How do school district characteristics currently reported relate to student 
academic achievement results? 

As in the 1988-89 study, a correlation matrix (Appendix B) was generated to 
assess the relationship between each reported characteristic and MSOs. However, 
four sets of relationships could be determined for 1991 : one for Elementary School 
Outcome Level (EOL), one for Middle School Outcome Level (MOL), one set for High 
School Outcome Level (HOL), and one for the System Outcome Level (SOL). The 
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Table 6. A Report Card Profile of 1 21 Tennessee School Districts sampled, 1 988-89 
data. 



A 0«* 

1 i 1 SCHOOL DISTRICTS 


3Q 


n 


Max 


Min. 


Report Card 
Mean 


OUTCOMES 












BaSIC Skills rirst (Bbr) 






( Percent passing 


: 8th grade 


Reading 


4.9 


121 


91 


65 


81 


Math 


7.7 


121 


85 


43 


66 


Stanford (STAT); Task 2 






(Stanine scored 


12th grade 


Reading 


0.5 


A f\A 

121 


7 


4 


5 


Math 


0.5 


121 


6 


4 


5 


English 


0.D 


Af)A 
1<1 1 


7 


4 


5 


Science 


0.5 


121 


6 


3 


5 


Social Studies 


0.5 


121 


5 


4 


5 


TN Proficiency Test 






f% Students Passing: 9th arade 


Language 




1?1 


98 


56 


76 


Math 
ivicui i 


6 4 


121 


98 


59 


90 


Roth 

DUll 1 




121 


98 


48 


76 


MONEY 












oo./oapiia income utytyon 






19,318 


6,934 


12,878 


Stud. Expenditure ($)(EPP) 


509 


121 


4,891 


2,318 


3,304 


Aver. Prof. Salary ($) (APS 


2,693 


121 


34,797 


21,286 


26,756 


SCHOOL SYSTEM 












Average Daily Mem.(#) (ADM) 
Student Attendance (%SA) 
Oversized Class (%OC) 


12,395 


121 


104,788 


375 


5,87 4 


1.3 


121 


97.1 


90.3 


93.6 


4.1 


110 


21.5 


0.1 


3.8 


Free/Reduced Lunch (%FRL) 


14.5 


121 


86.0 


9.0 


42.0 


Career Ladder 11/11 1 (%CL) 


5.9 


121 


41.5 


4.1 


14.8 



same correlation matrix (see Appendix B) displays relationships between independent 
variables and system outcomes (SOL). 

In response to question 1, Appendix B shows correlations exceeding ±.50 
between four district characteristics and LOL : percent of free or reduced lunches (r= - 
.70), percent of upper career ladder professionals (r= .62), percent of special education 
diplomas (r= -.53), and percent of Chapter 1 students (r= -.68). Five characteristics 
correlated above ±.50 with MOL : percent of free/reduced lunches 
(r= -.69), percent of upper Career Ladder teachers (r= .65), average professional 
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salaries (r= .51), percent of Special Education diplomas (r= -.69), and percent of 
Chapter 1 students (r= -.69). High correlations (above ±.50 ) existed between HOL and 
five district characteristics: percent of student attendance (r= .53), percent of 
free/reduced lunches (r= -.69), percent of upper Career Ladder teachers (r= .55), 
percent of special education diplomas (r= -.55), and percent of Chapter 1 students 
(r= -.74). When academic outcomes (MSO) for the entire system were the focus, four 
system characteristics demonstrated correlations above +.50: percent free/reduced 
lunches (r= -.73), percent of upper Career Ladder teachers (r= .64), percent special 
education diplomas (r= -.62), and percent of Chapter 1 student (r= -.73). 

Academic outcomes at all levels were influenced positively by the presence of 
expert teachers (upper Career Ladder teachers) and to a somewhat lesser degree by 
attendance. Attendance most influenced HOL performance. Most severe negative 
influences on academic performance at all levels were percent of students receiving 
free/reduced cost lunches and percentage of Special Education and/or Chapter 
1 students. 

A second set of data relating to question 1 (see Table 7 and Appendix C) 
provided a Guttman's Partial Correlation matrix for each of the four outcome levels and 



Table 7 Guttman's partial correlation used to evaluate the 15 report card 

categories from 4 educational levels (elementary (EOL), middle school 
(MOL), high school (HOL), and system (SOL), 1990-91 Tennessee school 
district report card data. 

EOL 0.2 6Tl 6/7 (X3 3^4 7.3 11.2 OA 32. To il (X3 61$ 15 2.1 39.60 

MOL 0.0 0.2 5.9 0.3 2.8 2.3 8.1 0.0 4.9 0.3 1.9 1.5 1.0 0.1 6.0 35.30 

HOL 4.5 5.3 13.6 3.5 0.2 0.3 0.4 0.6 0.0 2.7 0.2 0.4 4.5 0.0 4.7 40.90 

SOL 0.4 0.9 13.3 1.5 3.1 4.7 9.4 0.4 3.1 0.1 1.5 0.2 2.9 0.2 6.5 48.20 
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for the 1 5 targeted system characteristics, and a display of the percentage of influence 
exerted by each system characteristic on each set of MSOs. Findings included: 

1 . Characteristics having greatest impact on student academic performance were 
not the same at all levels (see Figure 1). The factor most influencing the EOL 
was per pupil expenditure (11.2%). Middle school student academic 
performance was most impacted by the same factor (8.1%). Academic 
performance among high school students was most influenced by Iheir 
attendance (13.6%), as was overall academic performance in the school district 
(13.3%). 

2. The factor having least impact on MSOs also varied by school level. The size of 
the system (ADM) had least influence on elementary student performance 

(0. 1 %). Neither the number of schools in a system nor the county per capita 
income had any influence on MOLs (0.0%). HOL was least influenced by the 
percentage of Special Education students in the district and the percentage of 
Career Ladder II and III teachers teaching there (0.0%). Overall MSO in a 
system was least impacted by average professional salaries of educators (0.1%). 

3. Percentage of oversized classes, a rough indicator of the influence of class size 
on student performance, has increasingly less influence on student academic 
performance as students progress from elementary to middle to high school. 
Even at its most influential point (the elementary years), this factor accounts for 
only 3.4 percent of whatever it is that influences student academic outcomes. 

2. Are there differences in the relationships between dependent and 
independent variables at different school levels? 

The answer to this question is clearly "yes" as demonstrated by data in Appendix 
C. We have already reported the differences in system characteristics having most and 
least impact on student academic outcomes at the various school levels, tto system 
characteristic influences student academic outcomes in the same way at all 
school/district levels. The combined set of 1 5 characteristics does not exert the same 
amount of influence on MSOs at any of the four levels studied. This finding will be 
explored more completely in response to research question 5. 

Other relationships demonstrated in Appendix C are important. The presence of 
upper Career Ladder teachers appears to have greatest impact on student performance 
at the middle school level (4.9%). The average professional salaries paid within a 
school district do not have great influence on student performance, but they have more 
influence (2.7%) on secondary students than on any other group. The socio-economic 
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level of the community (county per capita income) had less than one percent influence 
on academic outcomes at any level. 

The histogram presented in Figure 1 portrays the statistics presented in the 
Appendix C. Note particularly the positions of the influence occupied by percent 
student attendance (%SA), expenditure per pupil (EPP), and percent of students 
receiving free/reauced lunches (%FRL) in relationship to the other variables. 

3. Are there differences in the relationships between dependent and 
independent variables at different grade levels (2nd, 3rd, 4th, etc.)? 

The matrix presented in Appendix D displays the percentage of influence of each 

community/school characteristic on MSO at each grade level in 1990-91 , second 

through eighth and tenth. Similar to the school-level analysis, no. characteristic exerted 

the same level of influence on MSO at every grade level. The characteristics having 

the greatest impact on MSO across grade levels {mean of grade level percentages) 

were expenditure per pupil (6.0%), percent of student attendance (4.4%), percentage of 

free/reduced lunches (3.5%), and percentage of Career Ladder teachers (3.6%). 

Characteristics exerting the least impact on MSO across grade levels were number of 

schools in the district (0.4%), average daily membership (0.4%), average professional 

salaries (0.9%), and percentage of enrollment change (0.9%). 

Only five characteristics exerted six percent or more of all influence on MSOs at 

any grade level: percentage of student attendance, percentage of free/reduced lunches, 

expenditure per pupil, percentage of Career Ladder teachers II & III ,and percentage of 

Chapter 1 students. Of these five characteristics, only three exerted that level of 

influence (6 percent or more) on MSO at more than one level. Student attendance 

accounted for 13.2% of MSO at second grade and 6.6% percent MSO at 7th grade. 

Percentage of free/reduced lunches produced 8.5% of the influence on MSO at fourth 

grade and 11 .3% at the fifth grade. Per pupil expenditure exerted substantive influence 

at the third (11.7%), fourth (8.2%), fifth (10.6%), and seventh (6.4%) grade levels. 

Analysis of Appendix D data indicates some shifts in influence exerted by a 

single district characteristic from one grade level to the next. For example, the 

influence of percentage of student attendance dropped from 13.2% in the second 

grade to 3.5% in the third, influence of free/reduced lunches fell from 11 .3% in the fifth 
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Figure 1 The percentage of influence for the 15 report card categories and the four levels 
(elementary [EOL), middle [MOL], high school [HOL], and system (SOL], 1990-91 
Tennessee report card data. 
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grade to 2.8% in the sixth grade. A rise from 1 .6% to 11 .7% in the influence of 
expenditure per pupil appeared between grades two and three, and percentage of 
Career Ladder II & III teachers had far less influence on third grader's MSO (0.4%) than 
on second grader's (15.8%). 

When influence of all 15 district characteristics studied on individual grade-level 
performance is summed, the combined influence varies from a high of 48.3% at fourth 
grade to a low of 19.4% at sixth grade. As in previous analyses, less than 50% of all 
influence on MSO at any grade level is produced by these 1 5 factors. Performances of 
fourth graders and second graders are influenced most by the combined set of factors 
(48.3% and 39.9%, respectively). Sixth grade and tenth grade MSOs are least 
influenced by this set of factors (19.4% and 19.5%, respectively). 

Differences in the relationships between independent and dependent variables 
are found at different grade levels. Equal treatment of schools and clas srooms at even/ 
level does not appear to be the most appropriat e way to improve student performance. 

4. How do reported school characteristic relate to each other? 

The answer to this question is found in Appendix B. The correlation matrix 
reveals eight correlations exceeding ±.50. The relationship between number of schools 
in the system and %SA is strongly negative (r= -.54). The same can be said of the 
relationships between %SA and size of school district (n= -.54) and between %FRL and 
%SA (r= -.54). None are surprising statistics. 

There is a strong positive correlation (r= .53) between percentage of special 
education diplomas awarded in a district and the percentage of students receiving 
free/reduced cost lunches. A strong positive correlation (r= .78) exists between 
percentage of Chapter 1 students in a school district and percentage of students 
receiving free/reduced cost lunches. Special education, free/reduced meals, and 
Chapter 1 are closely linked. 

The relationship between APS in a system and EPP is strongly positive (r= .79). 
Communities that spend more on education pay their teachers and administrators 
better than do other communities. A strong positive correlation (r=.51 ) is found 
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between percentage of students receiving special education diplomas and percentage 
of students not receiving diplomas. 

There is a positive correlation (r=.50) between percentage of special education 
diplomas awarded and percentage of students enrolled in vocational education 
programs. This correlation could reflect the creation of vocationally-oriented programs 
for special education students, or placement of special education students in vocational 
programs, regardless of the suitability of the programs to the students. 

5. When rank ordered on the basis of student outcomes, how do school 
districts within the state perform in terms of reported school and 
community characteristics? 

To explore this question, the investigators generated rankings by MSO at the 
four levels being investigated and by system characteristics for the top 10 and bottom 
10 producing systems, using system MSOs (SOL) as the anchor. Table 8 and 
Appendix E present the findings. Table 8 displays the relationships between SOLs and 
school levels. Among important findings are the following: 

1 . The system having the highest MSO (#72) had the highest elementary and 
middle school MSOs, but not the highest high school MSO. 

2. Eight of the top 1 0 systems ranked by district MSO were not in the top 1 0 at the 
elementary, middle, or high school levels. 

3. The district ranking 1 0 in SOL ranked 60th in HOL performance. 

4. No district ranking among the bottom 1 0 districts in district MSO ranked above 
94th position at any school level. 

Appendix E provides data about school district/community characteristics in 
relation to system level MSO rankings. It also profiles the relationships between 
system/community factors and HOLs. Note the following: 

1 . There are no readily identifiable patterns of school/community characteristics 
among those currently reported that produce high achieving or low achieving 
school systems. 

2. There are no common patterns of school/community characteristics among 
those reported that appear consistently to produce high achievement or low 
achievement among high school students. 

3. Typical biases about characteristics necessary in a system or community to 
produce high achievement (e.g., money, larger or smaller schools, small 
classes) are not confirmed by the data available. Schools and communities with 

18 



Table 8 School District Rankings By Student Academic Performance, 1 990-91 
data, based on SOL and compared on EOL, MOL, HOL. 





Elementary 


Middle 


High School 


System 


Differences 




SCH 


EOL 


Rk 


MOL 


Rk 


HOL 


Rk 


SOL 


Rk 


Max. 


Mln. 


Dlff. 


# 


z 




z 




z 




z 




z 


Z 














Top 10 Systems 










72 


2.87 


1 


2.96 


1 


1.61 


4 


2.48 


1 


2.96 


1.61 


1.36 


119 


2.09 


3 


2.14 


2 


1.25 


11.5 


1.83 


2 


2.14 


1.25 


0.89 


59 


1.96 


4 


1.98 


3 


1.36 


9 


1.77 


3 


1.98 


1.36 


0.62 


84 


1.58 


13 


1.98 


5 


1.68 


3 


1.75 


4 


1.98 


1.58 


0.40 


QQ 


1.71 


"7 

i 


1.98 


4 


1.49 


5 


1.73 


5 


1.98 


1.49 


0.49 




1.71 


~t 

1 


1.17 


14 


1.78 


2 


1.55 


6 


1.78 


1.17 


0.60 


110 


1.71 


7 


1.19 


12 


1.42 


6.5 


1.44 


7 


1.71 


1.19 


0.52 


37 


1.58 


13 


1.82 


7 


0.85 


19 


1.42 


8 


1.82 


0.85 


0 96 


103 


1.71 


7 


1.18 


13 


1.29 


10 


1.39 


0 

%* 


1.71 


1.18 


0.53 


29 


2.22 


2 


1.83 


6 


-0.03 


S5L5 


1.34 


10 


2.22 


-0.03 


2.25 












Bottom 10 Systems 










97 


-0.75 


94 


-1.09 


112 


-0.75 


106 


-0.86 


111 


-0.75 


-1.09 


0.34 


16 


-0.88 


106.5 


-0.43 


98 


-1.36 


115 


-0.89 


112 


-0.43 


-1.36 


0.93 


62 


-1 .40 


115.5 


-0.60 


107 


-0.82 


108 


-0.94 


113 


-0.60 


-1.40 


0.80 


46 


-0.88 


106.5 


-1.25 


115 


-1.02 


112 


-1.05 


114 


-0.88 


-1.25 


0.37 


10 


-1.14 


113 


-1.72 


117 


-0.90 


109 


-1.25 


115 


-0.90 


-1.72 


0.81 


58 


-0.88 


106.5 


-1.09 


113 


-1.82 


117 


-1.26 


116 


-0.88 


-1.82 


0.94 


78 


-1 .79 


117 


-1.25 


114 


-1.43 


116 


-1.49 


117 


-1.25 


-1.79 


0.53 


41 


-1 .79 


118 


-2.53 


119 


-0.75 


107 


-1.69 


118 


-0.75 


-2.53 


1.78 


111 


-2.43 


119 


-2.54 


120 


-2.72 


120 


-2.56 


119 


-2.43 


-2.72 


0.29 


30 


-3.21 


120 


-2.52 


118 


-2.70 


119 


-2.81 


120 


-2.52 


-3.21 


0.69 



a range of the characteristics currently reported produce both higher and lower 
academic achievement. 

6. Do the reported school district characteristics appear to represent all or 
most factors that influence student academic achievement? 

The answer to this question is found in Appendix C. Clearly, the answer is, 

"NO." Together, the 15 characteristics under investigation provide 39.6 percent of the 

influence on EOL, 35.3 percent of the influence on MOL, 40.9 percent of whatever 

influences HOL, and 48.2 percent of the influence on SOL. These factors influence 

outcomes at different levels in different ways, and together they account for less than 
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half of whatever influences student performance at any level. Further, they account for 
less that 50% of the i, 'luence on student outcomes at any single grade level as 
indicated in the response to question #3. 

7. Is there evidence of major change in student academic performance from 
one school level to another within school districts? 



Table 9 provides the data pertinent to this question. Eleven systems 
demonstrated shifts downward in MSO of at least one standard deviation somewhere 
between the elementary and the high school levels. Sometimes the shift occurred from 
elementary to middle school, sometimes from middle to high school. Sometimes the 
change was continuous from level to level, and sometimes a dramatic shift occurred 
from elementary to middle, but began to reverse from middle to high school. 

Twelve systems demonstrated changes of at least one standard deviation 
upward over the three school levels. Again the patterns of change were not always 
constant, and the shifts occurred at different points in different systems. 

Some of the notable change patterns can be seen in reviewing the changes in 
academic rankings within a system from level to level: 

1 . Six of the 1 1 systems showing downward shifts in MSO had consistent 
downward trends from the elementary to middle to high school levels. 

2. Three districts showed significant declines in MSO from the elementary to 
middle school level, but reversed the trend from middle to high school. System 
#82 demonstrated a dramatic downward shift from elementary to middle school 
(20th to 78th) and a dramatic shift upward from middle to high school (78th to 
18th). 

3. Two districts (#71 , #9) displayed better student performance (by rank) at the 
middle school level than at the elementary level, but dropped markedly in the 
high school rankings. 

4. Of the 1 2 systems demonstrating upward shifts in MSO, six showed consistent 
patterns of improvement at each school level. Perhaps the most dramatic 
pattern was exhibited by system #1 which ranked 106 (of 120) in EOL, 23 in 
MOL and first in HOL. Data for this system also clearly point up the limited value. 
of district-level rankings . In the composite, this system ranked 28th in SOL. 

5. Three systems (#41 , #74, #52) displayed downward patterns of achievement 
from elementary to middle school, but strong upward patterns from middle to 
high school. 
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Table 9 School districts with outcomes greater/smaller than +<1 .0 z-scores between the 
elementary, middle, or high school levels. 



Elementary Middle High School System Differences 



SYSTEM 


EOL 


Rk 


MOL 


Rk 


HOL 


Rk 


SOL 


Rk 


Max. 


Min. 


Diff. 


# 


Z 




Z 




Z 




Z 




Z 


Z 














DOWN 


















AT Least -1 Standard Deviation at some tevel 


(n=H) 








101 


1.06 


20.5 


-1.52 


116 


-0.69 


102 


-0.39 


85 


1.06 


-1.52 


-2.58 


29 


2.22 


2 


1.83 


6 


-0.03 


60.5 


1.34 


10 


2.22 


-0.03 


-2.25 


22 


1,45 


16 


0.55 


21 


-0.67 


100 


0.44 


30 


1.45 


-0.67 


-2.11 


77 


■0,10 


57 


-0.28 


73 


-1,97 


118 


-0.78 


107 


-0.10 


-1.97 


-1.86 


85 


1,71 


7 


1.66 


8 


-0.15 


67.5 


1.07 


13 


1.71 


-0.15 


-1.85 


89 


1.58 


13 


0.20 


42 


&Q2 


58 


0.58 


24 


1.58 


-0.02 


-1.60 


82 


1.06 


20.5 


-0.43 


78 


0.94 


18 


0.52 


26 


1.06 


-0.43 


-1.49 


71 


1.58 


13 


1.33 


11 


0.09 


54 


1.00 


1Z 


1.58 


0.09 


-1.49 


67 


1.19 


18 


-0.11 


52 


0.42 


35.5 


0.50 


29 


1.19 


-0.11 


-1.29 


9 


1.58 


13 


1.48 


10 


0£9 


42 


1.12 


11 


1.58 


0.29 


-1.29 


39 


0.54 


30.5 


-0.43 


79 


■0.59 


94 


-0.16 


02 


0.54 


-0.59 


-1.13 














UP 


















At least +1 Standard Deviation at some level fn=12) 








1 


-0,89 


106.5 


0.54 


23 


1.85 


1 


0.50 


28 


1.85 


-0.88 


+2.73 


41 


-1.79 


118 


-2.53 


119 


-0.75 


107 


-1.69 


118 


-0.75 


-2.53 


+1.78 


74 


0.54 


30.5 


■JLZL 


69 


1,37 


8 


0.55 


25 


1.37 


-0.27 


+1.64 


90 


-1.40 


115.5 


-0.76 


110 


0*19 


48 


-0.66 


104 


0.19 


-1.40 


+1 .58 


55 


-0,88 


106.5 


-0.43 


97 


0.47 


33£ 


-0.28 


Z4 


0.47 


-0.88 


+1.35 


64 


-0,49 


76.5 


QM 


20 


-0.36 


78 


0.00 


57 


0.84 


-0.49 


+1.33 


51 


-0,88 


106.5 


0.39 


26 


-0.51 


89.5 


-0.33 


78 


0.39 


-0.88 


+1.27 


52 


:fi£2. 


82.5 


-0.59 


102 


0.63 


245 


-0.20 


08 


0.63 


-0.62 


+1.25 


33 


-0,82 


82.5 


-0.27 


66 


2A5 


£9 


-0.11 


58 


0.55 


-0.62 


+1.17 


93 


-0.49 


76.5 


0.21 


40 


0.64 


23 


0.12 


47 


0.64 


-0.49 


+1.14 


47 


-1,01 


111.5 


-0.43 


99 


OJ25 


50 


-0.46 


91 


0.05 


-1.01 


+1.06 


31 


-0.36 


70 


0.22 


37 


0.63 


24.5 


0.16 


43 


0.63 


-0.36 


+1.00 



KEY: 

SYS = State System ID 
EOL = Elementary Outcome Level 
MOL = Middle School Outcome Level 
HOL = High School Outcome Level 
SOL = System Outcome Level 
Bold = Unusuai data 
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6. Three systems (#90, #64, #51 ) showed strong upward trends in MSO and 

ranking from the elementary to middle school level, but reversed the pattern from 
the middle to the secondary level. 

The data presented do not suggest the courses of the changes found among 
these 23 school districts. Changes could relate to the quality of instruction at the 
several levels. They might reflect an emphasis on "teaching to the test" at certain 
levels. They could indicate the lack of alignment between outcome measure (tests) and 
curriculum. They might be caused, in part, by the movement to a new set of tests 
(TCAP) during the year being investigated. What is clear is that outcome data and 
rankings reported at the system level have limited utility in identifying what is happening 
academically within a system or in targeting areas for improvement. 

8. When academic achievement is treated as scores on two separate test 
batteries (TCAP and TPT), are patterns of influence changed? 

Appendix F presents the findings pertinent to this question. Percentages of 

influence of each school district characteristic on each high school student achievement 

measure (TCAP, TPT) were compiled. The high school TCAP score used because it 

represents the 10th grade level, the level closest to the point (9th grade) where the TPT 

is administered. Several statistics are noteworthy: 

1 . The combined influence of the 1 5 factors varies greatly from test to test 
(TCAP=19:5%, TPT=41 .6%). 

2. Student attendance plays a much more important role in passage of the TPT 
(13.7%) than in the scores attained on the TCAP (3.0%). 

3. Oversized classes influence TCAP scores (1 .3%) more than passing TPT 
(0.1%), but the influence is not great in either case. 

4. Size of the school district (number of schools and ADM) has more influence on 
TPT scores (5.5%, 5.1%) than on TCAP scores (0.9%, 0.6%). 

The difference in what is being reported in the two scores may have significant 

impact on the influence patterns. The TPT results are simply a summary of the 

percentage of students receiving scores of 70 percent or better on ali sub-tests 

(criterion-referenced). TCAP results reported are school-level mean scores on the test 

(norm-referenced). At any rate, various factors in the school district do influence 

outcomes on these two measures differently. 
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9. What differences in relationships among variables exist when test results 
of special education students are included in the analyses? 

Data appearing in Appendix G provide the response to this question. When 
rankings of the top 25 and bottom 25 performing school districts with special education 
students 1 TPT scores included were compared with the rankings for same districts 
excluding special education results, there were some changes in rankings, but no 
district originally ranked in the top or bottom group moved out of that respective group. 

Shifts in ranked position were both upward and downward. Few were dramatic; 
i.e., shifts did not change rank by more than a position or two. Among the top 25 
districts, one district dropped six positions when special education students' scores 
were dropped from consideration. Another district rose six positions under the same 
circumstances. Among the bottom 25 districts, three climbed markedly in rank when 
special education results were removed. Two districts dropped more than four 
positions. In large part, special education students 1 test results did not dramatically 
influence the overall academic performance of the school district. 

10. How do the results on this study compare with the results of the 
investigation using 1988-89 Report Card data? 

Results of the two studies (1988-89 and 1990-91) are not comparable in several 
areas. Student outcome measures (tests) changed in the interval, and the 1990-91 
report cards provided more and somewhat different data than in 1988-89. 

Changes and similarities in the basic statewide system profiles have already 
been presented in the descriptive analysis of school districts (see pp. 2-5). Therefore, 
the comparisons presented here focus on findings in response to similar research 
questions in the two studies. 

The 1988-89 study reported positive correlations between school district MSO 
and five school district characteristics: county per capita income, average professional 
salaries, per pupil expenditure, student attendance, and percentage of upper Career 
Ladder teachers. In that study, two district characteristics (%OC and %FRL) correlated 
negatively with student academic performance, and one characteristic (ADM) 
demonstrated no significant correlation to student outcomes. 
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In 1990-91 , system MSO (or SOL) correlated positively with the five district 
characteristics: student attendance, per pupil expenditure, county per capita income, 
average professional salaries, and percentage of upper Career Ladder teachers. The 
same two district factors that correlated negatively with student performance in 1988-89 
{%OC and %FRL) demonstrate that relationship again 1990-91 . in the 1990-91 study, 
size of school district (ADM) also demonstrated a negative correlation with academic 
performance. Relationships among variables change little from test to test or year to 
year (correlation data can be found in Appendix B, and in Bobbett, French, and 
Achilles, et.al. 2 ). In 1988-89, correlations exceeding .50 (±) were found among four 
sets of system characteristics: 

CCI and APS, r=.71 APS and EPP, r=J8 

CCI and ADM, r=.53 
CCI and %FRL, r= -.53 

When examining the same district cnaracteristics using 1990-91 data, three 
correlations exceeding .50 (±) were found: 

%SA and ADM, r= -.54 APS and EPP, r= .79 

%SA and %FRL, r= -.54 

Only one pair of characteristics {average professional salaries and expenditure per 
pupil) exhibited essentially the same relationships in the two studies. However, many of 
the positive and negative correlations below ±50 were exhibited from study to study. 

One comparison available from the two studies is the influence on MSO of the 
eight school district factors studied in 1988-89 and the 15 factors investigated in the 
current study. Table 10 presents the data. 

What produces the changes in influence of various factors is unclear. However, 
several observations can be made. 

1 . Attendance is still the most dominant factor in student achievement, among 
factors available for study. 

2. In both studies, factors considered by many to be major contributors to or 
inhibitors of student academic performance (e.g., teacher salaries, percent 
oversized classes, county per capita income) by themselves have limited 
influence. 

24 



3 J 



Table 10. Comparison of Influence Exerted On Student Academic Outcomes By 
School District Characteristics in 1 988-89 and 1 990-91 . 



District Characteristics 

County Per Capita Income 
Average Professional Salaries 
Expenditure Per Pupil 
Average Daily Membership 
% Student Attendance 
% Oversized Classes 
% Free/Reduced Lunches 
% Career Ladder II & III 
Number of Schools in District 
% Enrollment Change 
% Regular Diplomas 
% Honors Diplomas 
% students enrolled in 

Vocational Education 
% Students in 

Special Education 
Percentage of Chapter 1 Students 



Percentage of Influence (district level! 



1988-89 


1990-91 


0.4 


0.4 


o.b 


U. 1 


0.0 


9.4 


O Q 
C.O 


u.y 


10.9 


13.3 


0.6 


3.1 


6.0 


4.7 


0.2 


3.1 




0.4 




1.5 




1.5 




0.2 




2.9 




0.2 




6.5 



Total Percentage of Influence 26.5 48.2 
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3. Doubling the number of factors included in the analysis almost doubles the 
amount of influence for which one can account, but the 15 characteristics under 
scrutiny in the current study still account for less than half of whatever influences 
student performance. 

4. The change in student outcome measures from study 1 to study 2 may have 
significant impact on the data. If so, the importance of test/outcome measures 
themselves is underscored again. 

IV. CONCLUSIONS AND IMPLICATIONS OF THE STUDY 

Several of the conclusions the 1988-89 study were reinforced by the results of 
the 1990-91 investigation. Specifically, policymakers at all levels need to consider that 
few of the individual inputs commonly associated with student achievement have much 
impact on student performance. With the exception of student attendance (and 
perhaps per pupil expenditure) treatment of any isolated variable will have little effect. Jf 
we want to improve or change stude nt performanr.fi. a systemic approach to education 
change is an absolute necessity. 

In the 1988-89 study, the researchers concluded that the eight system 
characteristics taken from the Tennessee Report Cards for analysis were of limited 
value; i.e., they give limited information to policymakers and educators who want to 
improve education in their states and local communities, because these variables 
accounted for so little of the influence on student outcomes. In the 1 990-91 study, 15 
variables were examined. Again, they do not appear to be the "right ones," i.e., they 
don't tell us enough about what influences student achievement. Based on the two 
studies, knowledge gained from review of related research and experience in schools, 
the investigators urge that consideration be give n to collectino. reporting, and analyzing 
data on such things as school organization, school culture , student motivation, parental 
involvement, instructional methodologies, curriculum features and o ther factors to try to 
find factors that mav have significant influence o n student performance. 

When reviewing the results of the 1988-89 study, the investigators suggested 
that building-level data are probably more useful and m ore valid than district-level data 
for use in report cards. That conclusion is confirmed by the present study. Major 
variations and fluctuations in results appeared from school level to school level and 
grade level to grade level within individual school districts. Identification of sources of 
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these differences could be useful to educators and policymakers seeking improvement. 
Even the 1990-91 study did not have building-level data available for analysis. School- 
level data in the study may reflect conditions across several schools. 

Report cards are only as good as the assessments used to determine student 
performance . The 1 988-89 study raised some questions about the assessments being 
used. Those questions are highlighted in the findings of the current study. Enough 
variations in similar analyses between the studies exist to suggest that the differences 
in student outcome measures are probably one cause. 

The numerous variations found in influence patterns from grade level to grade level and 
school level to school level provide a great deal of food for thought. Much support has been given to 
the notion that schools and learners within a school district should be treated equally; i.e., per pupil 
expenditures, class size, quality of teachers and other factors should be the same in all situations. 
The findings of this study indicate that some factors are more important to student achievement at 
some levels than at others. £qual treatment may actually promote educational inequities within a 
school district. 

Finally, "What is the purpose of School District Report Cards?" The question is 
not an antagonistic one, but a supportive one. Definition of purpose or purposes is 
central to assessing the value of report card contents. A recent editorial in the 
Nashville Tennessean (1992) speaks A Tennessee's report cards in glowing terms: 

It (the Report Card) is simply the most comprehensive report in this 
or any state on school funding and student performance. . . 

The reports are more than just a tool for comparison, however; they 
can empower local communities to act. The reports give Tennesseans the 
power to get the job done and make the grade for better schools, (p.40). 

If the purpose of the Tennessee Report Card is simply to report the status of a 
community's schools and selected factors generally associated with them, the current 
report card may get by. If the purpose is to provide citizens, parents, educators and 
policymakers meaningful information upon which to make decisions for improvement, 
much is lacking. At best, at least 50 percent of what influences student performance 
has not been reported. This can provide serious impediments to school improvement, if 
education leaders focus entirely on what is now being reported as the primary sources 
of improvement in student performance. 
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Appendix A 



Results of Kaiser Test of Variable Sampling Adequacy 

20 report card variables 





MSA 


.226 




.228 




.230 


.230 




EOL 


.34 

OCX 


MOL 


.46 


HOL 

I.I tk hi 


,51 SOL 


.44 


1 


#SCH 


.31 


#SCH 


.27 


#SCH 


.28 #SCH 


.30 


2 


ADM 


.30 


ADM 


.27 


ADM 


.28 ADM 


.30 


3 


%SA 


.27 


%SA 


.46 


%SA 


.46 %SA 


.36 


4 


%EC 


.24 


%EC 


.38 


%EC 


.41 %EC 


.36 


5 


%OC 


.35 


%OC 


.67 


%OC 


.45 %OC 


.64 


6 


%FRL 


.29 


%FRL 


.24 


%FRL 


.25 %FRL 


.25 


7 


EPP 


.24 


EPP 


.41 


EPP 


.40 EPP 


.34 


8 


CCI 


.13 


CCI 


.10 


CCI 


.10 CCI 


.10 


9 


%ES 


.19 

1 Mr. 


%ES 


.17 
iff, 


%PS 


.16 %ES 


.16 


10 


%HS 


17 
1 1 1 


q/oHS 


.14 


%HS 


.15 %HS 


.14 




%CL 


.24 


%CL 


.20 


%CL 


.20 %CL 


.20 




Ml O 




APS 


44 


APS 


.46 APS 


.41 


13 


D-HS 


.18 


D-HS 


.15 


D-HS 


.16 D-HS 


.16 


14 


D-HO 


.19 


D-HO 


.17 


D-HO 


.17 D-HO 


.18 


15 


P-SE 


x22 


D-SE 


JtQ 


D£E 


*2QD£E 


x21 


18 


D-CA 


J2 


D-CA 




D-CA 


JIDrCA 


Jl 


17 


D-NR 


x21 


D-NR 


Jfi 


D-NR 


.IS D-NR 


Jl 


18 


%VO 


J4 


%V0 


.50 


%V0 


x2a %V0 


ja 


19 


%SE 


Jfi 


%SE 


J5 


%SE 


■14 %SE 




20 


%CH1 


.27 


%CH1 


.26 


%CH1 


.27 %CH1 


.30 



Underline-bold * Tennessee Report Card Categories 
that did not pass the test ot "independence". 
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Appendix C 

Partial Correlations and Percent of Influence of 15 School District 
Characteristics On Mean Student Outcomes 
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Appendix E 



Profiles of School District Rankings by District Mean Student Outcomes 
and School District Community Characteristics 
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Hiah School Outcome Level (HOL) 

Top12HOL Districts 



!l5 = <a8£8:oy£5l|^| 

1| 1 104 102 15 70 29 5 21 7 i M 92 2 25 

2 108 71 68 54 12 29 10 27 23 19 5 15 4 106 17 



1 


1 


104 


102 


15 


70 










7" 




44 










2 


108 


71 


68 


54 


12 




29 


10 


27 


23 


19 


5 


15 


4 


106 


17 


3 


84 


60 


45 


54 


1 




29 


1 


11 


4 


2 


117 




45 


114 


4 


4 


72 


94 


62 


20 


44 


17 


7 


14 


21 


3 


6 


61 


24 


34 


25 


22 


5 


99 


5 


4 


49 


13 


74 


5 


69 


4 


77 


16 


104 




47 


30 


2 


6 


110 


83 


91 


9 


41 




32 


16 


35 


2 


13 


104 




33 


10 


54 


7 


54 


104 


104 


4 


39 


36 


40 


78 


72 


32 


54 


76 


29 


3 


39 


14 


8 


74 


113 


1C5 


3 


32 


32 


51 


110 


72 


26 


48 


16 


36 


28 


91 


74 


9 


59 


37 


34 


41 


5 


4 


64 


2 


18 


6 


3 


4 


4 


97 


84 


30 


10 


103 


8 


10 


45 


83 




35 


7 


18 


74 


17 


50 


58 


53 


113 


40 


11 


119 


13 


12 


49 


9 


29 


1 


24 


1 


18 


20 


50 


45 


43 


28 


1 


12 


91 


28 


19 


32 


77 


87 


25 


103 


C6 


28 


41 


81 


7* 


52 


49 


20 














Bottom 12 HOL Districts 
















109 


10 


18 


26 


120 


92 


98 


110 


61 


100 


100 


84 


13 


56 


110 


58 


114 


110 


83 


71 


64 


38 


72 


44 


101 


89 


109 


32 


94 


89 


78 


30 


74 


110 


111 


88 


104 


113 


54 


91 


69 


91 


52 


78 


116 


107 


108 


99 


112 


110 


52 


112 


46 


83 


50 


15 


82 




116 


70 


86 


89 


75 


33 


32 


65 


5 


116 


113 


21 


71 


99 


23 


112 


27 


60 


76 


69 


35 


82 


89 


85 


76 


42 


99 


114 


92 


83 


57 


106 


118 


48 


91 


62 


58 


61 


79 


38 


41 


46 


52 


99 


11* 


16 


30 


44 


112 


107 


91 


107 


54 


75 


92 


103 


61 


71 


99 


89 


113 


116 


78 


1 


1 


118 


98 


49 


117 


13 


4 


51 


12 


23 


91 


73 


8 


105 


117 


58 


50 


86 


117 


114 


74 


112 


33 


116 


105 


119 


29 


5 


102 


100 


107 


118 


77 


83 


101 


84 


119 


97 


91 


65 


101 


9 


55 


26 


6 


40 


115 


42 


119 


30 


60 


43 


90 


102 


46 


119 


90 


63 


117 


113 


61 


95 


50 


47 


120 


120 


111 


94 


82 


114 


53 


103 


96 


117 


106 


90 


109 


100 




78 


71 


103 



33 



Appendix F. 



Guttman's Partial Correlation (r2) Used to Compare Influence of 
Community/School Variables on TPT and TCAP Results. 
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Appendix G 



Profile of School District Academic Performance With and Without 
The Inclusion of Special Education 
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